CI: Push Linux Build to Launchpad

In .travis it is possible to initiate the push of a Linux build to your Launchpad repository. These instructions assume that you have a travis account and know your way around the services and basic process. Your travis account is used to confirm 'successful' builds of linux and osx, and to create Linux packages to deploy to your Launchpad.

For travis refer to Petri's post #637

1. You can use this OCPN_Draw_pi .travis file as an example. The important part is the “- provider: launchpad” section.

Example Bottom section of the travis.yml file. Note: indentation is important in yml files, child parts MUST be indeted below the owner. This is shown below by “oauth_token:” being the owner and “secure:” being the child, notice that “secure:” is indented.

deploy:
  - provider: launchpad
    slug: "Your repository slug here"
    oauth_token:
      secure: "Your encrypted oauth_token here" 
    oauth_token_secret: 
      secure: "Your encrypted oauth_token_secret here"
    skip_cleanup: true
    on: # Set Deploy conditions
      # Deploy only when tag is not specified
      # tags: false
      # or this branch
      #branches:
      #   only:
      #     - master
      tags: true
      all_branches: true
      condition: "$TRAVIS_OS_NAME = linux"

The “slug:” and “secure:” data items are in double quotes (“”).

If you want another deploy section, like git, you will need to create a new “- provider:” with the text “releases” following it. Both “- provider:” labels need to have the same indentation.

1. You need a travis CLI (command line interface) to be able to encrypt the Launchpad tokens. The encrypted strings are put in your personal .travis.yml file.
2. Travis CLI installation instructions for Windows. When you are sure that you have travis CLI up and working, you can start to create the encrypted key. Without it, deployment to GitHub does not work.
3. Travis CLI installation instructions for Linux. Windows10 users can create a WSL-Windows Subsystem for Linux (Ubuntu 18.04LTS) on Win10 Installation, and then from the linux command prompt execute:

   sudo apt-get update --fix-missing
   sudo apt-get install ruby
   sudo apt-get install gem
   ruby -v
   gem install travis -v 1.8.10 --no-rdoc --no-ri
   travis version
   response: 1.8.10.
   
   225mb disk space used.

Part of the process requires the http “Post” process to acquire security tokens. The easy way to do this from a command prompt using the “curl” program. * For Linux: Your package manager should be able to install “curl” as it is a standard internet tool. * For Windows: goto this website http://techieroop.com/how-to-install-curl-on-windows-10/ and select the correct version for your setup.

Issue the following command from a command prompt.

curl --dump-header - -H "Content-Type: application/x-www-form-urlencoded" -X POST --data 'oauth_consumer_key=Travis+Deploy&oauth_signature_method=PLAINTEXT&oauth_signature=%26' https://launchpad.net/+request-token

You should receive a reply like:

HTTP/1.1 200 Ok
Date: Sat, 06 Jul 2019 01:57:06 GMT
Server: zope.server.http (HTTP)
X-Powered-By: Zope (www.zope.org), Python (www.python.org)
Content-Length: 132
X-Xss-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Strict-Transport-Security: max-age=15552000
Vary: Cookie,Authorization,Accept-Encoding
X-Content-Type-Warning: guessed from content
X-Launchpad-Revision: 19007
X-Frame-Options: SAMEORIGIN
Content-Type: text/plain;charset=utf-8

oauth_token=XJwTXv9f8RRRT6TwBW4x&oauth_token_secret=lbrXHsQ0jVlJlqLpNX4mGWtNjcRp2rWzqhQx53KDhTM87m7f7MMGTNw7DsG3CcbhSbn3jwTCmMkFbFBF

Save this information as you will need it in following steps.

You will now need to logon to your Launchpad account. You need to have been authenticated before you can issue the next command from your browser. Open a new browser tab and enter the following in the “Address line”:

https://launchpad.net/+authorize-token?oauth_token={your oauth_token from step D:}

Your version could look like:

https://launchpad.net/+authorize-token?oauth_token=XJwTXv9f8RRRT6TwBW4x

You will now be asked for the level of access to provide this token. The easiest is to select “All”, the last button on the page.

In your command prompt window issue the following command:

curl --dump-header - -H "Content-Type: application/x-www-form-urlencoded" -X POST --data 'oauth_signature=%26{your oauth_token_secret}&oauth_consumer_key=Travis+Deploy&oauth_token={your oauth_token}&oauth_signature_method=PLAINTEXT' https://launchpad.net/+access-token

Your version would look like“

curl --dump-header - -H "Content-Type: application/x-www-form-urlencoded" -X POST --data 'oauth_signature=%26lbrXHsQ0jVlJlqLpNX4mGWtNjcRp2rWzqhQx53KDhTM87m7f7MMGTNw7DsG3CcbhSbn3jwTCmMkFbFBF&oauth_consumer_key=Travis+Deploy&oauth_token=XJwTXv9f8RRRT6TwBW4x&oauth_signature_method=PLAINTEXT' https://launchpad.net/+access-token

From this you would get a response like:

oauth_token=DJZmQkJ65F3M0HD4Lvr5&oauth_token_secret=WbwMzlDhW3HpkHxLh7wrtkXqPZkFS8n6HHXrZXLj4WK0vKljjJqHtdx9hWmCLXJLZ0CQSc5kD8jWnscm&lp.context=None

You should now encrypt your tokens before they are used. This makes the interaction more secure.

• Change directory to your local code base for git.
• Issue command:

travis encrypt {your oauth_token from step F.> --org

• This may look like:

travis encrypt DJZmQkJ65F3M0HD4Lvr5 --org

• You will receive a response like:

Please add the following to your .travis.yml file:

  secure: "O+F3qadTm1EojsoGlopwLPirEJ8TkshlyZGOUdOa62YJsI7SrE3ugryeU2wuYHRBxE0dhXDCxHkUf+E45yq4PL8IIHx29eoygsZeVwHCgyT2nN/Pk/O/EOJXeCRo5WgrliyDnHCl4Dmz7vyRiP6svWuzz1xM6ATpKXSUlb02tdzcoydD0bzeUYwUnK2VRc7yv4Afxgr0yXx+7fdscV+ALCZ2JOvroxXWGRJawo96KWEE0On72/wiYZXeMuC7QtxFBQopSRNrmzGMTHv9F8c6cGT/Utcrp86xw42JO0pxOXCdd5i+60hzlA6QWswrNIuNBVnYWydGfJoX56bcalRZ1MEE+NyU/zHYBi8uehh8NIfUAYxdY7w7VW6FtyEdayNGWBm1FRGyyOquYfFQMGMHCBrnt30IKk2glSFWLxUBUTec4c3GGOXO8W0Zq/IhGE2ej77AVzCXgB1zOiruUSJdpkpQH73xC174SQBmiW54k76V4oRy2YyIXeibC04zKVhSpfJ0ORObvbZbz4LAgx0ML4emxFUFsIkzCNK872rEPDiUJjcL6So3bNKTrV0RC44uXN+JzFXrAeHwCZVATXANg4Ah4N94YX4W5sk0iNArhN6/p50kZbffxvZnYlPIh728BvtCFkGi5IELEeFt0Q/wE1BUr6k7ozxhGJtSTohaCa0="

Pro Tip: You can add it automatically by running with --add.

PS. Don't do as suggested with the –add as it may destroy your indentation in your .travis.yml file. It is far better to select the line starting with “secure:” and copy paste it into the requisite position in the .travis.yml file.

1. You will need to turn the project “on” after signing into your travis account. Click on the icon in the top right with the down chevron and select 'Settings', the page will display your respitories under the 'Repositories' tab. Make sure the ones you want to use have the slider button to the right with a tick in the button. If it is to the left and shows a cross it is not active.
2. Remember to change VERSION.cmake patch number and date, an commit and push that too! This is important if you are going to create a release as this will help identify the build objects belonging to the release.
3. Then when you commit .travis.yml file, the travis build will start. It is good to check that both of its build processes do pass faultlessly before the next step.
4. It takes awhile for each commit (Linux and Mac) to build correctly, be patient when it is running.
5. Make sure it built successfully. The project will not be saved because travis does not provide that type of storage space.

1. Now the project is ready to create a Release in Github. There are two ways to do this.
2. Option 1: Create the new 'Tag' from within Github Release Tab.
   • In GitHub create a repository, like v1.2.3 as 'tag', description can be anything, but it best to include the version. This is where Travis and AppVeyor both deploy the built contents.
   • When you hit the create button, Travis and AppVeyor will create the 'build' and deploy (the condition flags in the configuration file are define this action) to the Github 'Release' Tag.
   • That's why it's important to check before you create the 'Tag' that all builds do pass faultlessly. (Repositories cannot be deleted since they are, in fact defined by tags.)
   • If you observe 401 error at the deployment phase in travis logs search in google.com with the following phrase for possible causes “travis github https://api.github.com/user: 401 - bad credentials”
3. Option 2: Command Line - Make a 'Tag' locally and push it to your remote.
   • From your command line create a new tag locally, and then push that tag to your remote repository.
   • All the same cautions apply, make sure it builds properly first.

      OPTION 2: TAG COMMANDS  Make tag locally and push to remote
      git tag v1.13.1    <--Create the tag on your local repository
      git push origin v1.13.1  <--Push the tag to your remote repository, travis creates build and pushes.
      git push –delete origin v1.13.1  <--You've made a mistake, it fails to build, remove remote tag.
      git tag –delete v1.13.1  <--You've made a mistake, it fails to build, remove local tag.
      Fix the problem and start over.

When the Travis build does not work the error messages can be obscure and not really help too much. A typical message may be:

Skipping a deployment with the provider because this branch is not permitted:

This is not very helpful as it just says something went wrong not what.

1. Check your whole .travis.yml file. Look at it carefully and compare it to the example given.
2. Make sure you have the security token enclosed in double quotes, i.e. “security/token”
3. Make sure the build process has actually worked for the file types you are trying to transfer to github
4. If you have extra values in your yml file make sure you really know what they do. If you have issues, take all the extranous values out and see if it works then.
5. If necessary find a very small project and play with that as it will compile much quicker.
6. Make all changes in a branch of the main repository as you will have to do multiple commits to get it working. Once it is working you can make the changes to your main branch(es) and clean up the 'test' branch

The use and configuration of Appveyor is similar to Travis in many respects. We use Appveyor to create Windows packages.

# Artifacts Configuration
artifacts: # push all files in directory
  path: build\*.exe
  name: installer

# Deploy to GitHub Releases
deploy:
# description: 'release created by AppVeyor CI'
  provider: GitHub
  auth_token: # '%GitHub_auth_token%'
    secure: VVAVg9a...[put the appveyor encryption of your github public token here ]...f1OSYg0tS
  artifact: installer
  draft: true
  prerelease: true
  tag: $(APPVEYOR_REPO_TAG_NAME) # use pushed Tag or insert version name
  on:
    configuration: Release  # Debug contains non-redist MS DLLs
    APPVEYOR_REPO_TAG: true   # deploy on tag push only
#   branch: master   # release from master branch only

1. You can also use this Weatherfax_pi appveyor.yml file for reference and there are some in process notes available.
2. Login to your Appveyor Account and Create a Github API Public Security Token under New Personal Token, entering a “Note” similar to Appveyor_auth_token_[plugin_name] and then select a Permissions “Scope” for “REPO” by checking “Repo”.
3. Copy the resulting code to your clipboard and somewhere else, as this is the last time you will have access to the public key.

1. Sign into the Appveyor Encryption Service and paste the public key into the field.
2. Push “Encrypt”. Copy the result to clipboard.

1. Then paste that into the appveyor.yml file after auth_token: and secure:, where the words <encrypted GitHub API token here> occur. The encrypted public token can be enclosed by quotes.
2. Save the appveyor.yml file,

1. You will need to select or turn the project “on” after signing into your appveyor account.
2. Then when you commit appveyor.yml file, the appveyor 'build' will start. It is good to check that its build processes do pass faultlessly before the next step. Commit and push to the remote repository.
3. Additionally you must remember to change VERSION.cmake patch number and date, an commit and push that!

   git add appveyor.yml
   git commit -a -m "update appveyor.yml"
   git push origin master

1. It takes awhile for the commit (Windows) to build correctly, be patient when it is running.
2. Make sure it built successfully by checking that the Artifact Tab for the install package file.
3. Now the project is ready to create a Release in Github. There are two ways to do this.
4. Option 1: Create the new 'Tag' from within Github Release Tab.
   • In GitHub create a repository, like v1.2.3 as 'tag', description can be anything, but it best to include the version. This is where Travis and AppVeyor both deploy the built contents.
   • When you hit the create button, Travis and AppVeyor will create the 'build' and deploy (the condition flags in the configuration file are define this action) to the Github 'Release' Tag.
   • That's why it's important to check before you create the 'Tag' that all builds do pass faultlessly. (Repositories cannot be deleted since they are, in fact defined by tags.)
   • If you observe 401 error at the deployment phase in travis logs search in google.com with the following phrase for possible causes “travis github https://api.github.com/user: 401 - bad credentials”
5. Option 2: Command Line - Make a 'Tag' locally and push it to your remote.
   • From your command line create a new tag locally, and then push that tag to your remote repository.
   • All the same cautions apply, make sure it builds properly first.

      OPTION 2: TAG COMMANDS  Make tag locally and then push to remote

Examples:
git tag v1.9.1-ov42    <--Create the tag on your local repository
git push origin v1.9.1-ov42  <--Push the tag to your remote repository, travis creates build and pushes.
git push –delete origin v1.9.1-ov42  <--You've made a mistake, it fails to build, remove remote tag.
git tag –delete v1.9.1-ov42  <--You've made a mistake, it fails to build, remove local tag.
Fix the problem and start over.

1. The git push origin v1.9.1-ov42 Tag command is executed.
2. Upon which a new Release & Tag “v1.9.3-ov42-test” gets created in GitHub “Releases”, which now contains the results of the - - Appveyor build under the “Assets” label. You should find a file similar to “weatherfax_pi-1.9.3-ov42-win32.exe”.
3. See https://github.com/rgleason/weatherfax_pi/releases/tag/v1.9.3-ov42-test
4. The Appveyor 'Console Tab' shows: https://ci.appveyor.com/project/rgleason/weatherfax-pi
5. The Appveyor “Artifacts Tab” shows the file pushed to Github 'Release': https://ci.appveyor.com/project/rgleason/weatherfax-pi/build/artifacts

     Collecting artifacts...
     Found artifact 'build\weatherfax_pi-1.9.3-ov42-win32.exe' matching 'build\*.exe' path
     Uploading artifacts...
     [1/1] build\weatherfax_pi-1.9.3-ov42-win32.exe (721,873 bytes)...100%
     Deploying using GitHub provider
     Creating "v1.9.3-ov42-test" release for repository "rgleason/weatherfax_pi" tag "v1.9.3-ov42-test" commit "05b6418e674b7d722424146c8efe2745a88b635b"...OK
     Uploading "weatherfax_pi-1.9.3-ov42-win32.exe" to release assets...OK
     Build success

G. How to get to these Appveyor Locations? The simplest way to access Appveyor is through Github commits eg:https://github.com/rgleason/weatherfax_pi/commits/master

• If appveyor succeeds in creation of an Artifact - green check.
• If appveyor succeeds in creation of an Artifact and a Release (if a new Tag is pushed to remote.) - green check.

Note that VERSION.cmake entries/changes and TAG must be manually coordinated!

For more information about setting the auth_token security please read notes in Appveyor Forum discussions:

• Push built Artifact .exe to Github "Release" (opensource, public repos, dev account) Post 13 by fcgleason on Apr 23, 2019 @ 09:31 PM
• See Git branch --delete and Git Squash -Effect on Appveyor

It just updates the patch number and updates the Appveyor file so that we can push a new tag to the remote repository and then appveyor will push the windows artifact into the Release (new tag). (This just requires the setup of a personnal auth_token and copying the Yaml encrypted version of that into the appveyor file.)

It's a nice efficient way to make a release with an identical tag. This is what I've done on my repository:

• Post #2396
• Post #2047
• Post #2425
• Use of appveyor 'auth_token' to push a new github release #134
• Getting auth token

: